fix(image-index): release service resources after shutdown - #9911

Summary

Release image-index-owned encoders and vocabulary/search matrices after shutdown, once both the worker's resource work and admitted requests have drained.

stop() currently signals and joins the worker but keeps these strong references. Clearing only in stop() is unsafe: the worker can outlive its bounded join, and an admitted search can populate the cache after shutdown returns. Track worker drain explicitly and let shutdown, the exiting worker, or the final request perform cleanup only after both owners drain. An unconditional worker finalizer also handles exceptional exits without swallowing BaseException.

Keep the existing ten-second join bound, shared model-manager caches and arrays still held by callers. No reaper thread, polling, global cache eviction, or change to the two-entry LRU.

Related Issues / Discussions

Addresses the shutdown-retention portion of #9632. The separate scope-cache sizing question remains open; this PR intentionally does not close the whole issue.

QA Instructions

Base: current main de172fa7685ced4394c0e6ac4716f0a63fe8751e.

The 12 added regression cases use tiny real CPU modules/NumPy arrays, real SQLite reads, event-gated worker/request threads, and weak references. They cover both drain orders, late search-cache fills, a worker surviving the bounded join, worker SystemExit, final requests raising RuntimeError/SystemExit, repeat shutdown, caller-held arrays, and shared-cache preservation. Join expiry is simulated without a real ten-second wait.

Local results:

• Exact upstream service replayed in memory, without changing tracked files: 12 failed, 3 existing controls passed. Failures are resource-retention/cache-release assertions, not import or timeout failures.
• Candidate shutdown regressions plus those controls: 15 passed.
• Complete image-index suite, image-map routes and database-access architecture guard: 351 passed, with 16 dependency/API deprecation warnings.
• Project-pinned Ruff 0.11.2 check/format and git diff --check: passed.

Owning-suite commands (from the repository root, with application/temp/cache directories isolated and BLAS threads capped at one):

python -m pytest tests/app/services/image_index/test_image_index_service.py -o addopts= -p no:cov -n 0 -q -k 'stop_without_a_worker_releases_owned_encoder_resources or stop_releases_owned_encoder_resources or stop_defers_encoder_release_until_inflight_request_finishes or stop_releases_search_matrix_populated_by_a_late_request or stop_defers_resource_release_past_worker_join_timeout or stop_waits_for_both_worker_and_request_resource_users or stop_does_not_evict_shared_model_manager_caches or stop_preserves_a_search_matrix_still_owned_by_its_caller or stop_releases_resources_when_last_search_request_raises or stop_joins_worker or stop_releases_a_worker_waiting_for_request_users_to_drain'
python -m pytest tests/app/services/image_index tests/app/routers/test_image_map.py tests/app/test_database_access_guard.py -o addopts= -p no:cov -n 0 -q
ruff check invokeai/app/services/image_index/image_index_default.py tests/app/services/image_index/test_image_index_service.py
ruff format --check invokeai/app/services/image_index/image_index_default.py tests/app/services/image_index/test_image_index_service.py
git diff --check

The executed runs explicitly loaded the xdist/AnyIO plugins with plugin autoload disabled, used offline model settings, and used the current project's direct dependency versions. Initial environment-only import failures were resolved by installing the current pinned packages; no production workaround or weakened test assertion was added.

This establishes service ownership release, not an observed 600 MB RSS reduction. No physical GPU, complete-model generation, full-repository build/suite, comprehensive typecheck, or local MySQL/MariaDB validation is claimed.

Review

No material findings remain.

The deliberate tradeoff is deferred cleanup while an owner outlives shutdown's bounded join, rather than freeing resources still in use or evicting another service's shared cache. Ordinary retirement errors retain their existing logging behavior; worker exceptions still propagate.

AI-assisted tooling was used for implementation, regression development and current-main integration. The reported checks were actually executed.

Compatibility / Rollout

No persisted-data, public API, generated-schema or cache-capacity change. Existing stop/replacement admission and the ten-second join limit remain. Caller-held arrays stay valid, and shared model-manager cache ownership is unchanged.

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Meaningful regression coverage added / updated where needed; obsolete tests/code removed
• Persisted-state and API changes include required migrations / compatibility validation (not applicable: neither changed)
• Relevant performance/efficiency opportunities considered; material claims have evidence (ownership release, not RSS/throughput claims)
• Material review findings resolved and relevant checks rerun
• Documentation added / updated (if applicable) (lifecycle ownership and lock constraints documented in code)
• Updated What's New copy (if doing a release after this PR) (not a release PR)

^{Need help on this PR? Tag @codesmith-bot with what you need.}